EXPONENTIAL APPROXIMATION OF EMPIRICAL NEIGHBORHOOD 
MEASURES OF RANDOM GRAPHS BY RANDOM ALLOCATION 



^ " By Kwabena Doku-Amponsah 

II 

Abstract 

In this article we correct a conjecture in |DM06a| Remark 3] and, improve the proof of 
the large deviation principle for empirical neigbourhood measure of coloured random graphs 
conditioned to a given empirical colour measure and empirical pair measure. Using this 
result, we found a large deviation principle for the empirical degree distribution of an Erdos- 
Renyi graphs, where nc edges are inserted at random among n vertices. We show that, for 
any degree distribution d, the solution to the variational formulation of the rate function in 
|BGL02| Theorem 7.1] is the relative entropy of d with respect to the Poisson distribution 
with mean c. 



1. Introduction 

1.1 The coloured random graph model. In this subsection, we review the coloured random 
graph model as in [DM06a]. We begin by fixing the following notations. Let X he a, finite alphabet 
or colour set X. Let V be a fixed set of n vertices, say V = {1, . . . , n}. Denote by Q n the set of all 
(simple) graphs with vertex set V = {1, . . . , n} and edge set E C £ := {(it, v) G V x V : u < u}, 
where the formal ordering of edges is introduced as a means to simply describe unordered edges. 

Given a symmetric function p n : X x X — > [0, 1] and a probability measure [i on X we may define the 
randomly coloured random graph or simply coloured random graph X with n vertices as follows: Assign 
to each vertex v G V colour X[v) independently according to the colour law \x. Given the colours, 
we connect any two vertices u, v G V, independently of everything else, with a connection probability 
p n (X(u), X(v )) otherwise keep them disconnected. We always consider X = ((X(v) : v G V),E) 
under the joint law of graph and colour. We interpret X as coloured random graph and consider X(v) 
as the colour of the vertex v. Denote by G n (X) the set of all coloured graphs with colour set X and 
n vertices. See, [GP03j or [Pe98] , 

Our interest in this article is on the coloured random graph models in the near- critical critical regimes. 
Thus, we look at cases when the connection probabilities satisfy np n (a,b) —> C(a,b), for all a, 6 G X, 
and C: X x X -> [0, oo). 

For any finite or countable set y we denote by A4(y) the space of probability measures, and by M(y) 
the space of finite measures on y and we endow it with the weak topology. Further, we denote by 
M*{y x y) the subspace of symmetric measures in M.(y x y). 
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For any coloured graph X = ((X(v) : v G V),E) with n vertices we recall the definition of the 
empirical colour measure L 1 G A4(X), by 

L x {a) := - ^2$x (v){a), for a G , 
n vev 

and the empirical pair measure L 2 G .M^A* x X), by 

£ 2 (a,&):=- 2 [^(X(v),x(u))+^(x(u),x(v))}{a,b), for a,beX. 

We observe that by definition I? is finite symmetric measure with total mass ||-L 2 || equal to 2\E\/n. 
Finally we recall the definition of the empirical neighbourhood measure M G M(X x Af(X)), by 

M(o, £) := - J2 S (x(v),m) ( a > for (°> ^) e ^ x 

where = (l"(b), b £ X) and Z"" (6) is the number of vertices of colour b connected to vertex v. 
For every v G A4(X x N(X)) let ^2 be the A'-marginal, respectively the M{X)— marginal, of the 
measure v. Moreover, we define a measure {v(-,£), £(■)) G A^A 7 x X) by 

(i/(-,^),i(-))(a,6) := J] u(a,£)£(b), for a,beX. 

Define the function $: A4(A- x M{X)) -> M(^) x A4(A" x AT) by $(1/) = (z/i, (z/(-,-Q, £(•))), and 
observe that f>(M) = (X , L 2 ), if these quantities are defined as empirical neighbourhood, colour, and 
pair measures of a coloured graph. Note that while the first component of $ is a continuous function, 
the second component is discontinuous in the weak topology. 

We call a pair of measures (w, v) G A4(X x X) x A4(X x M(X)) sub- consistent if 

€(-))(a,6) < m{a,b), for all a, b G A", (1.1) 
and consistent if equality holds in (jl.ip . 
For any n G N we define 

M n (X) := G A4(Af) : nu(a) G N for all a G AT}, 
A4 n (A- x X) := {w €M*(XxX) : 1+1{ n a=fe} w{a, b) G N for all a, 6 G A"} , 



1.2 The conditional coloured random graph models. Throughout the proof we may assume 
that uj{a) > for all a G X . It is easy to see that the law of the randomly coloured graph conditioned 
to have empirical colour measure uj n and empirical pair measure w n , 

P K>ra7n) =P{- \$(M) = (cJ n ,W n )}, 

can be described in the following manner: 

• Assign colours to the vertices by sampling without replacement from the collection of n colours, 
which contains any colour a G X exactly nui n (a) times; 

• for every unordered pair {a, b} of colours create exactly n(a, b) edges by sampling without 
replacement from the pool of possible edges connecting vertices of colour a and b, where 

, h \ _ i n ^n(cL, b) if a 7^ b 
m n {a,b) :- < n^^y if a = b 
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2. Main Results 



2.1 Exponential approximation by random allocation. In order to improve( shorten) the 
proof of Theorem 13.11 we pass to a simple random allocation model, which turns out to be equivalent. 
This model is best described in term of coloured balls being placed randomly into coloured bins. 

Fix n > 1, a colour law u n £ M. n (X) and an edge law zu n £ M. n (X x X). The bins V = {1, . . . ,n} 
are now coloured by drawing without replacement from the collection of colours, which contains the 
colour a £ X exactly nu n (a) times. For every ordered pair (a,b) £ X x X of colours, we independently 
place nw n (a,b)/2 balls of colour b into the nu> n (a) bins of colour a by drawing without replacement. 
We denote by 'P( u „,m n ) the distribution of the random allocation model with colour law u n E Ai n (X) 
and an edge law zu n £ M. n (X x X). 

In the resulting constellation we denote, for any bin v £ {1, . . . , n}, by X(v) its colour, and by l v (b) 
the number of balls of colour b £ X it contains. Now deine the empirical occupancy measure of the 
constellation by 

M(a,£) = -J2 <W (t>) L{v)) (a,i), for (a,£) GXx M(X), 

U veV 

where L(v) = (l v (b),b £ X) is the colour distribution in bin v. In our first theorem we establish 
exponential equivalence of the law of the empirical occupancy measure M under the random allocation 
model P( Wn)n7n ) and the law of the empirical neighbourhood measure M under 

Pk w „)=P{" |*(M) = (o7 n ,w n )}, 

the law of the randomly coloured graph conditioned to have colour law uj n and edge distribution w n . 
Recall the definition of exponential equivalence, see [DZ98, Definition 4.2.10]. 

Theorem 2.1. The law of M under P( aJnjron ) is exponentially equivalent to the law of M under 
r k,n„) • 

We define the degree distribution D £ A1(N U {0}) of the coloured random graph by 

D ( k ) = Y, E forA:£NU{0}. 

aex teN{x) 

Theorem 2.2. Suppose D is the degree distribution of an Erdds-Renyi graph, where [nc] edges are 
inserted at random among n vertices. Then D satisfies a large deviation principle, as n — )• oo, in the 
space A4(NU {0}) with good, convex rate function 

8(d) = I H ^ q ^ (2.1) 
[ oo otherwise. 

where q x is a Poisson distribution with parameter x. 

Here we remark, that the LDP result of [BGL02, Theorem 7.1] holds and the conjecture that [BGL02, 
Theorem 7.2] does not hold is false. In fact the coupling argument of [BGL02J and Bennet's inequality, 
see |Be62| . proves Theorem 7.1 of [BGL02]. We define the function I by 



Theorem 2.3. 



( Jq H(r]t || rj t )dt, if r\ is absolutely continuous, , ^\ 

1 oo otherwise. ' 

inf {/(r?) : V eD M ([Q,2c}, M(NU{0}), t? 2c = d} = 5(d). 
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Remark 1 The proof of this theorem follows from the LDPs for the degree distribution of an Erdos- 
Renyi graph, where [nc] edges are inserted at random among n vertices, due to Theorem 12.21 and the 
results of [BGL02I Theorem 7.1]. 

3. Proof of Theorem 12.11 Theorem 12.21 and Theorem 12.31 

3.1 Proof of Theorem l2?Ll 

Define the metric d of total variation by 

d{y,v) = \ Yl \u(a,£)-u(aj)\, for u,u G M(X x M{X)). 

As this metric generates the weak topology, the proof of Lemma |2. II is equivalent to showing that for 
every e > 0, 

lim ±logP{d(M, M) > e) = -oo, (3.1) 
where P indicates a suitable coupling between the random allocation model and the coloured graph. 
Proof. 

To begin, denote by V(a) the collection of vertices (bins) which have colour a G X and observe that 

(JV(o) = nu n (a). 

For every a,b G X, begin: At each step k = 1, . . . , m n (a, 6), we randomly pick two vertices Vj 1 G V(a) 
and V 2 fe G V(b). Drop one ball of colour b in bin V* and one ball of colour a in V^, and link V* to 
V£ by an edge unless = V£ or the two vertices are already connected. If one of these two things 
happen, then we simply choose an edge randomly from the set of all possible edges connecting colours 
a and b, which are not yet present in the graph. This completes the construction of a graph with 
L 1 (X) = oj n , L 2 (X) = w n and 

d{M,M)<l £ B n (a,b), (3.2) 

a,beX 

where B n (a, b) is the total number of steps k G {1, . . . , m n (a, b)} at which there is disparity between 
the vertices Vj*, drawn and the vertices which formed the k th edge connecting a and b in the 
random graph construction. 

Given a, b G <Y,the probability that = V£ or the two vertices are already connected is equal to 

P[k]( a , b ) : = mn }a,b) ha=b} + (l ~ m „( a ,6) (Jn(a]b))' 2 ' 

B n (a,b) is a sum of independent Bernoulli random variables X\, -^ nron ( a ,b)/2 with 'success' proba- 
bilities equal to p m (a, b), . . . ,P[ nWn ( a ,b)/2] (a, b). Note that E[X k ] = p[ k ] (a, b) and 

Var[X k ] = p [k] (a,b)(l ~P[ k ](a,b)). 

Now, we have 

n(a,b) 

EB>, 6) = £ p w (a, 6) = i {a=6} + (l - ]l {a=b} ^y) (l - ^y) < 1 + i {a=6} . 
fc=l 

We write 

m n (a,6) 

fc=i 
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and observe that 

w?/T->nr„ i: t / t->tii _ r_ -l\ _2 i _ -i, 

Ha=fe} 



lim E(S n (a,6)) = lim Far(£ n (a,6)) = lim ^(0,6)0^(0,6) = ls a=b} + 1. 



We Define /i(x) = (1 + x) log(l + x) — x, for x > and use Bennett's inequality, see [Be62j . to obtain, 
for sufficiently large n 

for any <5i > 0. Let e > and choose <5i = p, Suppose that we have B n (a,b) < 5. Then, by f|3.2[) . 

d(M(jr),i/„) < 25im 2 = e. 

Hence, 



(d(Af,M) > e} < ^{B n (a,b) > n8i} < m 2 sup P{B n {a,b) > l {a=b} + 1 + {n5 x )/2} 



a,b€Pc ' 



< m 2 sup exp 



Let < #2 < 1- The, for sufficiently large n we have 

ilogP{d(M(X),M(X)) >e] < -(l-W^kj) 

^ 2(l {a=6} + l+5 2 )' llOS ^ ^ 2(H {a _ 6} +l+<5 2 )^ 2(l {a=b} +l+<5 2 )_ 



> n 



= -{l {a=b} + 1 - 5 2 
This completes the proof of the lemma. 



3.2 Proof of Theorem [2T2l 

As the degree distribution D is a continuous function of M, Theorem 13.11 below and the contraction 
principle imply a large deviation principle for D. This Theorem is a special case of Theorem 13.11 which 
was first proved in [DM06aJ abeit a number approximation steps, some of which are not too easy 
to understand. See, example }DM06al Lemma 4.10, p. 26-29]. Note, that in special case of classical 
Erdos-Renyi graph we have that M = D the degree distribution and 

(S(M)) = \E\fn = c. 

Theorem 3.1 (DM06a). Suppose the sequence (u} n ,w n ) G A4 n (X) x A4 n (X x X) converges to a limit 
(u,w) € A4(X) x A4*(X x X). Let X be a coloured random graph with n vertices conditioned on the 
event {<3?(M) = (u n ,zD n )}. Then, as n — > 00, the empirical neighbourhood measure M of X satisfies 
a large deviation principle in the space A4(X x N(X)) with good rate function 

~ ( \ _ j H{y || Q) if (zu, v) is sub- consistent and v\ = oj , . 

- j oo otherwise. ^ > 

Now Lemma |2. II and the large deviation principle for M under P( Wnjro?l ) implies the same large devi- 
ation principle for M under P( Wnjron ) in the weak topology. See, for example [DZ981 Theorem 4.2.13]. 
Consequently, the proof of Theorem 13.11 is equivalent to showing that for every V C M(X x J\f(X)), 
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Lemma 3.2. 

- inf J ( )(i/)< IiminfJlogP{MGr|*(M) = (w„,07 n )j 

< limsup i logpjM G T | $(M) = (cj n ,m n )\ < - inf jL w) (i/). (3.5) 

We begin the proof of (|3.5p by recalling the definition of $ a function on A4(X x J\f{X)) given by 
v i-> (fiiX^OM*^))- Let ( w n,^n) -> G 7W(<Y) x A4(^ x X) and write 

K,( n \uj n ,zu n ) = \v n : v n = M, $(f n ) = (u n ,zu n ), for some random allocations process on n bins }. 
We denote by S(u) the support of and write for v n G K.^> (u n , w n ), 

K,^) = nap(o; n ,t37 n ) - n/3^ n) (fn) - ^|5(^ n )| log27rn, 

where 

ai n) K,w n ) = -I log \KF>(u> n ,m n )\ + ± ^ logro n (a,6) + + \X\ 2 ) log2vrn 

+ £ E lWa)+l/n + ^ E lo S W ™( a ) + 7? 2 



l (a)+l/n ^ n ^ ^ 12tt7 n (o,6)+l/n ' 

a&X a&X a,b&X 



Pl\vn) = l E log^M) + ^ £ 



12i/„(a,£)+l/n" 

(a,()6*xAf(A') (a,^)eEA' xN{X) 

v n (a,e)>0 i/„(a,<?)>0 

We write ^\w n ,v n ) = naj (w n ,ro n ) — nfii^ ( u n), where 
c4 n) (u; n , ron ) = ilog|/cW(w n ,tJ7 n )| + 4 r ^ l2 J n{ab) + ^ T2^I)+^ E lo g ro ™(a,fe) 

a,6eAT a,6GA" 

aj n (a)>0 

+ + |*| 2 ) log 2™ + i J] log Wn (a), 

4 n) K) = ^ E k E 

(a,<)6afxA/'(Af) (a^)eEA' xAA(A') 

i/„(a,O>0 i/ n (a,£)>0 

We prove (|3.5p above from the following lemma based on the idea of the method of types, see, [DZ98 , 
Chapter 2]. 

Lemma 3.3. For any v n G JC^ n \uj n ,w n ), 

e -nH{u n \\Q n )+^\w n , Vn ) < p| M = ^ I $ ( M ) = ( Wn>tJ7n )} < |X:( n >(w n ,tJ7 n )r 1 e- nH ( Wn ll <? ^ +,, a" ) ( w »'^, 
w/iere 

Q„(a, £) =w n (a) H tttTj ,for£eN(X) 

hex ^ >' 



lim t^ nj (ro n ,z/ n ) = lim ^ n j (ro n , i/ n ) = 
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Proof. The proof of this lemma is based on the method of types, see, |DZ981 Chapter 2], combinatoric 
argument and good estimates from refined Stirling's formula. | 

We denote by x the random allocation process and observe that for any v n € JC^ n \oj n , w n ) we have 

{./»> Ia>rih I \\ »{^- M(i) = "n, »(M) = K,s;„)} .... 
P{M = u n $(M) = {u n: w n )} = t{i:(LHihLHi)) = iWn ^ n)] ■ 

Now, the right side of (|3.6p may be evaluated in the following way: 
• For a given empirical measure v n with. $(^ / ri ) — {j-^ni 

w n ) there are 
nuj n {a)) \ n / nzu n (a,b) 



TT ( nUJ n [a)) \ yr / 

11 U^(a,^), £ e N{X)J J = 1, ..., nWn (a) 

equally likely random allocation processes and 



• for every empirical and empirical pair measure $(^ n ) = (w n ,zz7„) there are 

/ \nro„(a,6) 

Tla be A" ( na; n( a ) J equally likely random allocation processes 

Therefore, (j3.6|) is equivalent to 

jP{M = i/ n |*(M) = (w n ,tJ7 n )} (3.7) 

n/ nw n (a)) \ -r-r / mu„(a,6) w 1 

^ Ui/„( 0| <), f G AT(^)i l^')^), j = 1, ... )nWB ( a )i ' [ ■ } 

while P{M = v n | <&(M) = (w n ,ro„)} = when ${v n ) ^ (u n ,vj n ) by convention. 
Suppose w n {a, b) = 0, for some a,b E A? then 



nw n (a, b) 
£ { J\b), j = l,...,nu n (a) 



1. (3.9) 



Suppose vj n (a,b) > 0, a good estimate of (raz7 n (a, &))! can be obtained from the refined Stirling's 
approximation, as 



exp [nw n {a,b)\ognw n {a,b) - nm n (a,b) + \ \ogw n {a,b) + 12rm7n ( a , fe ) + i + |log2n7rJ < (nw n {a,b))\ 
< exp (nw n (a,b) log nw n (a, b) - nw n (a,b) + \ logtZJ n (a, b) + 12n J n ^ + ^log27rn). 

Similarly, from the refined Stirling's approximation, sec [Fc67, p. 52], we have 



(2 7 r)2n n+ 2 e -™+ 1 /( 12 "+ 1 » < n! < (2tt) 2n" + 2e-" +1 « 12 "l, see [?, page 52]. 
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exp (n wn(o) log w„(o) - n zv n (a, log v n {a,l) + \ Yl n^+i/n + ~ lo § 27m ) 

aGA (a/) aGA 

XeXp ("« E iKM + i/n ) ^ II ( Wtl(a ,™f e V(^)) <exp(n£> n (a)log Wn (a)) 

( a ,e)exxM(x) aex ny ' y ' v 7 aGA 

i/„(a/)>0 

xexp(-n^^(a^)log^(a,^)-I £ + log 2nn + ± £ ^) . 

(3.10) 



(al) (a.i)exxM(x) aGA 

v n (a,l)>0 



We observe that U a ,bex II™^ &\b)\ = IL e * exp (n£ (a ,,)(log^)!H(a, b)), and hence 



exp ( $Z [ n $Z OTn ( a ' fo ) lo § ro n(a, b) - n Y ^n(a, b) - n ^ ro n (a, 6) logw n (a)] + ± ^ logro n (a, &)) 

6GA aGA aGA aGA 

e X p(»^^(logn6)!)>'„(o,<) + 1 ^ log2 Tn + £ 12 „„J„, M+1 ) 

£ n 



&GA aGA aGA aGA a,6GA 

X 

beX (a,i) a,b€X 

nZU n (a,b) \ / 1 \ nm n (a,b) 

^ { J\b), j = l,..,nw n (a)''™ n (a) 



a,f>GA 



< exp ^(log£(6)!)i/ n (a,£) + ± ^ log ro n (a, &)) 

bGA (a,£) a.ftGA 

x ex P (XI t n ro n(a,k)log X] w n{a,b) -n^2 w n (a,b) - ro„(a, 6) log w n (a)] + ^ i 2 nJ w (a,6) ) 

&GA aGA aGA aGA aGA a.ftGA 

x exp log 27rn^ . 

Putting everything together and choosing t?^(tz7 ra , u n ) and ^^(wr^Un) appropriately, we have that 



exp (nH(v n ) + ^ [n ^ ro n (a, 6) log ^ tu n (a, b) - n ^ ro n (a, 6) - n ^ ro„(a, 6) logw n (a) 

6GA aGA aGA aGA aeX 

x exp ( - nH(co) - n £ £(log*(&)!)i/ n (a, £) + ^ n) (n7n,^n)) < P{M = ^ n | <&(M) = (u n ,m n )} 

beX (a,e) 

< exp ( - nH(uj) - ^ [n y^(logl (b)\)v n (a, i) - n ^ Tu n (a,b) - n ^ ro n (a,6) logw n (a) 

ftGA (a/) aGA aGA 

x exp (nH(v n ) + n ^ ^ ro n (a, 6) log ^ tu n (a, 6) - log |/C (n) (w n , + ^ n) (ro„, f n )) . 

bGAaGA aGA 
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Collecting and rearranging terms properly and using &{v n ) = (ui n ,w n ), we have that 

H{u n ) - H{u n ) - [^(&KM)bg^(&KM) -J^(&)i/ ft (M) -Y,£(b)v n (a,e) log iu n (c 

beX (a,£) (a,t) (a,t) (a,i) 

-£(log*(&)!K(M)] 

= J2^(a,i)[logu n (a,i)-logu n (a) - £ (log(^f )^ - ^gf - log £(log£(6)!; 

(a,e) " bex e 

= J2"n(a,l)[logU n (a,l) - log {u n (a) J[ (^(a,6)/^(a))^(-t P „( a ,6)/^( a )) ) j 

(a,e) " bex 

= H(u n || Q n ) 

which completes the proof of Lemma 13.31 □ 

We prove from Lemma 13.31 and [DM06a, Lemmas 4.1 and 4.4], upper bounds and lower bounds in the 
large deviation principle for all finite n. Let V C M(X x J\f{X)). Then (|3.3|) gives the upper bound 

p{Mer|$(M) = K, ron )}= Y, P{M = v n \$(M) = (u n ,m n )} 

f„6rn/c( n )(w„,ro„) 

< V \JC( n \u n ,w n )\- 1 e- nH ( Un \\ Qn)+n ^ ( -" n '™ n) 

-ninf _ T ,_ r ( n )^ H(fn II QrO+ntfo™' (^»i,ro n ) /o-ii^ 

< g i'„6rnKi"»(u„,ra„) ^ ^ (3.11) 

The corresponding lower bound is 

p{Mer|$(M) = ( Wn ,© n )}= £ P{M = i/ n |*(M) = K,o7 n )} 

^ernK»(w„,ro„) 

-n inf . i?(i/„||Q„) v— v 

!/„ernx;c«)(a)„,u7„) 



-ninf _„_„f n -|, sH(u„ || Q„)+nt)j"'(u„,n„) , . 

> g ^„Grn)C | .'^(ujn,ro n ) " x . (3.12) 

Since limsup^^ ( w m ro n) = and liminf n -» ^'d{ l \cj n , zu n ) = ( by |DM06at Lem- 
mas 4.1 and 4.4]), the normalized logarithmic limits of (]3.1ip and A3. 12H gives 

limsup - logP{M G T I $(M) = (oj n , va n )\ = — liminf { inf H(v n \\Q n )\ (3.13) 

and 

liminf ilogp/M G r I *(M) = (w n ,tJ7 n )| = -limsup { inf fl> n || Q„)} (3.14) 

^ J n^cx. y„ern/c(")(£j„,-!37„) 

The upper bound in (|3.3p follows from (|3.13p , as T n /C^")(u; n , tu n ) C T for all n. 

Now fix i/ G M(X x J\f(X)). Then, by |DM06aJ Lemma 4.9], there exists a sequence f n G T n 
such that z/„ , — > ^ as n — )• oo. Therefore, by continuity of entropy, see, example [DM06a, 



10 



KWABENA DOKU-AMPONSAH 



p. 19, equation 14] we have that 

limsup { inf H(v j| Q n )\ < hm H{y n || Q n ) = H(u \\ Q). 

Recall that H{y || Q) = oo whenever, for some (a,£) € X xJ\f(X), u(a,£) > while Q(a,£) = 0. Hence, 
by the preceding inequality we have 

limsup { inf H{y ||Q n )}< inf H(v\\Q), 

n->oo v'ern/C(™)(aj n ,ro, l ) v&nt(T) 

which gives the lower bound in (13, 3j) . This completes the proof of Theorem 13.11 
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